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SIGNATORIES 



Many educators and researchers have proposed 
reforms for dealing with the ongoing adolescent 
literacy crisis. Progress in strengthening young 
people’s literacy now depends on schools a) choosing 
appropriate programs and b) implementing them 
consistently and effectively. Yet the history of school 
reform is littered with reforms that were adopted 
and quickly abandoned for not answering students’ 
needs. Time and resources routinely go to waste as 
schools and school districts recycle through reform 
after reform showing little long-term progress. Often, 
schools adopt reforms in a mechanical fashion, as 
if a perfunctory acceptance of the reform and its 
training requirements were sufficient to routinely 
yield educational success. But without appropriate 
leadership, resources, and commitment to succeed, 
any such effort is bound to fail. 

This paper reviews the literature on implementation 
of educational reforms, then examines differences in 
implementation and costs among a sample of schools 
that have each adopted one of three well-known 
(READ 1 80, Questioning the Author, and Reading 
Apprenticeship) reforms, concluding with a few 
recommendations — simple procedures that we believe 
will improve schools’ chances for achieving positive 
results with literacy reform. 

Importance of Implementation 

Educational reform at the school district or site 
typically begins with a specific challenge that 
has reached a crisis point, such as low student 
performance, disruption or absenteeism. Those in 
charge of fixing the problem usually survey a number 
of alternatives such as new textbooks, curriculum 
packages, software, and staff development, before 
making a choice. Soon afterward, materials are 
purchased and professional development begins, and 
the reform is said to be in place. Decision-makers 
begin to look for results within just a few months 


after starting the reform. After a year or so, when 
the expected improvements are still not forthcoming, 
they decide that the reform did not work and launch a 
search for a new and better one. 

This behavior is so typical of the reform process 
that anyone who has worked in a school or school 
district for a decade or so will have experienced as 
many as three or four different reforms dedicated to 
the same problem, and dozens of different reforms 
addressing the myriad issues facing their schools. 
School reforms tend to be adopted in a perfunctory 
manner, implicitly assuming that the adoption of new 
materials and professional development associated with 
the reform will automatically transform the school 
and alleviate the targeted problem. But such is almost 
never the case, so the search for solutions continues to 
follow a repetitive pattern as previous efforts fail and 
get replaced by a new approach. 

Two explanations are often given by observers for 
the familiar stop-and-go pattern of school reform. The 
first is that schools are unique institutions which cannot 
be altered by taking a cookie-cutter approach. Rather, 
change strategies must take account of the unique 
features of the school situation, including previous 
school experience with reforms, school leadership and 
level of commitment to change, staff capacity, student 
characteristics, and available resources. 

For example, schools with habitual adoption 
and turnover of reforms may see reform as a ritual 
rather than a reality to be taken seriously. School 
administrators may lack a full understanding of the 
needs of a reform and how to create productive roles 
that will support the reform. School staff may not 
believe that change is needed, and may attribute 
problems to factors such as teacher turnover, 
inadequate resources, and student limitations. 
Furthermore, schools may launch a reform without 
first gaining a clear idea of what resources are required 
and where they will be obtained. Likewise, school 
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districts may encourage or mandate adoptions of 
reforms without providing the supportive conditions 
for success. 

This explanation for failure of reform suggests 
that any strategy for reform must build not only on 
the theory and details of the reform, but also on the 
concrete features and realities of the school in which 
the reform is being enacted (Evans, 1996). 

The second reason often given for repeated failure 
of reforms is closely related to the first. According to 
this second view, not only does each school differ as a 
context for change, but even an understanding of the 
differences may not be fully adequate for developing 
a strategy for change. Schools are not quiescent or 
inert organizations waiting to follow instructions 
from outside experts on how to alter themselves. 
Rather, they respond to outside interventions, 
molding them in ways that are often unpredictable 
and even unrecognizable. In a famous set of studies 
by the Rand Corporation, it was found that although 
adopted reforms are explicitly designed to change how 
schools operate, schools themselves have the agency 
and wherewithal to reshape the reform, or even to 
neuter it completely. The failure of school reforms can 
therefore be attributed to the capacity of the schools 
to swallow external interventions without allowing the 
reforms to fundamentally change school directions. 1 

Because this second ex- 
planation is so closely related 
to the first, it suggests much 
the same strategy for mak- 
ing a chosen reform work, 
rather than permitting it to 
be derailed by aspects of the 
school culture into which it 
gets introduced: namely, de- 
tailed resource planning and 
cost-accounting prior to im- 
plementing the reform. 

Although there exists a massive literature on 
the importance of paying attention to the details 
of implementation (cf. Berends, Kirby, Naftel, & 
McKelvey, 2001), school organizations and operations 
typically undervalue the need for implementation 
planning. The Rand multi-year evaluations of the 
New American Schools (Berends et al., 2001) 
concluded that: “Throughout the history of research 
on program initiatives, one finding has emerged again 


and again: Implementation dominates outcomes” 

(p. 23; see also Fullan, 1991; Pressman & Wildavsky, 
1973). Schools need to plan in greater depth the 
process of implementing reforms, beginning with 
the specification of the types of resources that will be 
needed and their costs, and moving on to include the 
concrete, day-to-day details of implementation. 

This paper outlines how resources can be identified 
and their costs more precisely determined. By 
paying closer attention to the resource needs and 
uses necessary to support reform, rather than merely 
presuming that the reform will make itself work once 
it has been put into process, schools have a better 
chance of achieving their desired results, thereby 
breaking out of the endless cycle of hope, effort, and 
disappointment. 

Resources and Implementation 

Although there may be many reasons for poor 
implementation of school reform, two of the most 
common are (1) failure to account for the resources 
that will be needed to promise success, and (2) failure 
to procure the appropriate resources at the outset. 

Effective reform requires concrete efforts to 
ascertain the precise resources that will be needed to 
make the reform work. One cannot simply assume that 
these resources will automatically become available 


when and as needed. For example, the demand for 
reduced class size has specific resource consequences 
in that schools must hire more teachers and provide 
additional classroom space. Yet often, at a late hour, 
decision-makers find themselves scrambling for the 
necessary additional space and qualified personnel. 
Likewise, schools sometimes fail to make provisions 
for the time commitments of their personnel, 
overlooking or improperly scheduling necessary 
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work hours. Longer instructional periods for literacy 
activities usually means that provisions for teacher 
assignments and scheduling must be changed. If the 
school day remains fixed, resources and time must 
be reallocated, with obvious consequences for the 
overall scheduling of classes and assignments. In an 
extended day, additional teaching resources must be 
provided. Time for professional development includes 
not only the time allocated to formal workshops and 
professional development, but also time required 
for instructional planning among faculty, and 
consultation and evaluation by coaches with classroom 
teachers. Yet, all too often, schools make no formal 
arrangements to capture the time needed for such 
activities. Rather, they assume that the time will be 
“found” somewhere or other. If school principals, 
along with other school-site administrators, are to 
show real leadership in the reform process, they must 
find the time to engage in training, teacher meetings, 
classroom observations and feedback, modeling good 
practices, and so on. Yet, many decision-makers simply 
assume that administrative staff will find the necessary 
hours for all of these activities by rearranging their 
daily schedules to juggle the additional responsibilities. 
In some reforms, schools are expected to hire a 
full-time or part-time coach. Instead, the added 
coaching tasks are handed to a Title I coordinator or 
department head on top of existing responsibilities. 

Fortunately, there exist relatively simple tools 
for identifying the resources needed to implement 
reforms. These tools are easy to use because they 
build on knowledge of the resources and activities 
integral to the intervention. The basic model used to 
evaluate the resources that will be needed and their 
cost is what is known as the “ingredients method” 
(Levin & McEwan, 2001). The method requires that 
planners follow a number of relatively simple steps in 
planning implementation. 

The first step is to identify the “ingredients” or 
resources that will be required to put a reform into 
place (Levin & McEwan, 2001, Chapter 3). Such 
identification must be done in a systematic way that 
entails participation by both school and district staff 
members, all of whom need to understand exactly 
what it will take to implement the reform. Likewise, 
district staff must be fully cognizant of program 
requirements and funding needs. Most aspects of 
this type of analysis can be done by using a financial 


spread sheet such as EXCEL. On the spreadsheet, 
personnel positions are listed according to their 
qualifications and the portion of time that will be 
needed. If the principal is expected to allocate one- 
quarter of her time to the reform, that requirement 
is identified in a formal way rather than entrusted to 
“whether and when she has time.” The same goes 
for other personnel positions. During the planning 
process, it is crucial that decision-makers begin to 
identify where the needed personnel will come from. 
Will extra teachers be needed to free up time for 
professional development and teacher discussions 
and deliberations? If so, how many positions will 
be needed and with what qualifications? Clearly, 
the answers to such questions have far-reaching 
implications for hiring or reassignment. 

Facilities needs and specific furnishings and 
equipment should also be identified in the spreadsheet 
“ingredients analysis.” If additional classroom 
space is needed for reductions in class size, that 
space should be specified. If computers, software, 
instructional materials, and other equipment are 
required, these should be denoted explicitly rather 
than merely presumed. Ultimately, all necessary 
ingredients will be listed along with sufficient detail 
regarding qualities and characteristics. The compiling 
of the needed ingredients is not only important for 
developing a complete list of resources associated 
with the intervention; it also gives school and district 
personnel a better understanding of the needs and 
purposes of the overall reform, and enlists joint 
support in obtaining the resources. Specific sources 
of information in identifying ingredients include: 
descriptive materials on the reform and interviews 
with the sponsor or developer of the intervention; 
articles and reports on experiences of other schools 
in adopting the intervention; and observations 
and interviews (often by email or telephone) with 
personnel in other schools or districts that have 
already adopted the intervention. 

The second stage in using the ingredients method 
to identify resources and implementation needs is to 
associate each of the ingredients with its cost. Methods 
of setting out the cost for each have been well- 
developed in the literature (Levin & McEwan, 2001: 
Chapter 4). A complete listing of ingredients and their 
costs will provide an estimate of the overall cost of the 
intervention. This process also clarifies the resources 
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that must be in place to promise success, so that 
the school year begins with the necessary personnel 
and adjustments in schedule and group size that are 
integral to the specific reform. 

The third stage in the ingredients form of 
analysis is to ascertain where resources will come 
from. In some cases reallocation of the budget may 
be necessary, assigning existing resources to the 
intervention in place of using them for activities of a 
lower priority. In other cases new resources will be 
required with implications for school budgets or for 
obtaining volunteer support. At this stage the details 
of financing must be in place in order to move forward 
with the reform. 

Implementation is a joint responsibility of the 
model developer and the school and school district. 
The model developer must provide clear guidelines 
with respect to how the reform works and what 
modifications it requires to usual school practices as 
well as the ingredients needed to make it succeed. The 
school and school district must set out a blueprint in 
advance that allows for identification, funding, and 
acquisition of required resources and planning the 
activities that comprise the reform. The precise nature 
of the blueprint will vary from site to site depending 
upon initial alignment of programs and personnel. 

In some schools the challenge of a specific reform 
will require greater modifications of resources and 
prove more costly than in others. For example, as we 
will show, reforms that require a small class size are 
simpler to implement in a district that already has 
smaller classes than the norm. 

In the following pages we illustrate some 
dramatically different costs and methods of 
implementing the same reforms. Our aim is to assist 
school decision-makers and schools in selecting 
from among these or other interventions to improve 
literacy among their students based not only on 
students’ needs, but also on a careful consideration of 
the costs of implementation. In some of these cases, 
the wide divergence in costs is due to the need to 
make larger departures from existing practices at some 
sites than at others. In other cases, the site decision- 
makers have decided to make special modifications 
in the models. In still others, idiosyncratic factors 
seem to have entered into implementation decisions. 
The overall lesson we hope to convey is that careful 
planning and analysis in advance of launching a 


reform is likely to provide both better implementation 
and better cost management. 

Although we are illustrating the variability in 
implementation among a small sample of adolescent 
literacy reforms, the overall findings should not be 
viewed as unique. Virtually all reforms show this 
type of variability, even those largely implemented 
by “formula.” For example, one of the most widely 
used reading reforms at the elementary school 
level, Success for All (SFA), shows similar variability 
despite relatively rigid requirements in materials, 
organization, and instructional practices. Using the 
ingredients method of estimating costs, King (1994) 
found that Success for All had an implementation cost 
ranging from about $500 per child to about $1,300 
per child, even though its sponsors report a cost of 
only about $150 per child for materials and training 
(about $75,000 for a school with 500 students). 

The cost difference can be accounted for by the 
fact that SFA requires schools to provide extended 
class periods for reading, smaller class sizes, and 
additional personnel in the school for supporting the 
reform, changes that are not accounted for in the cost 
estimates of the developer. Deciding whether large 
differences in cost from site-to-site are associated 
with differences in effectiveness among sites lies 
outside the scope of this study, though that possibility 
certainly exists. 

Three Adolescent Literacy Programs 

To demonstrate the variability in implementation 
and subsequent variability in costs, we selected three 
highly regarded reforms for improving adolescent 
literacy. We then gathered information on how these 
programs were implemented at different sites. We 
collected data on the logistics of the implementation 
as well as the resources used to carry out the 
intervention at each chosen site. Each developer 
provided the same description of the intervention and 
its implementation requirements to the different sites. 
From the perspective of the developer, success requires 
that the nature of the professional development and 
the provisions necessary for the reform have clear 
similarities, ensuring quality control from site-to- 
site. We describe three interventions and explore 
differences in their implementation and costs among 
sites to see how much variance exists. In order, the 
chosen interventions are: 
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■ Read 180 

■ Questioning the Author 

■ Reading Apprenticeship 

Read 180 

Read 1 80 is a reading intervention designed for 
struggling readers in late elementary, middle and high 
school. Read 180’s stated goal is to improve students’ 
decoding, fluency and comprehensions skills. The 
program was developed through collaborative research 
between Vanderbilt University and the Orange County 
Public School System in Florida. It was piloted with 
more than 10,000 students between 1994 and 1999 
(Scholastic Research and Evaluation Department, 

2006, July). Scholastic began publishing Read 180 in 
1999 and currently markets the program to school 
districts across the country. 

Read 180 lessons consist of whole group, small 
group and individualized literacy instruction. During 
whole group instruction teachers read aloud, engage 
students in shared and choral reading, and model 
fluent reading and the use of reading strategies. 

The class is then divided into three groups that 
rotate through three reading stations: small 
group instruction, computerized instruction and 
independent reading. In small group instruction, the 
teacher gives more personalized reading instruction 
to a small group of students. At the computer 
station, students receive individualized instruction 
via a program that advances to new text only after 
students demonstrate mastery in fluency, word 
recognition, spelling and comprehension. The 
program provides support for readers, including 
a video to enhance background knowledge, 
pronunciation, translation and definitions for 
difficult words in the text, decoding tips, and a 
summary of the student’s reading accomplishments. 
During independent reading, students self- 
select texts from the Read 180 library and listen 
to audiobooks, which model fluent reading and 
comprehension strategies. The Read 180 lesson ends 
with another short period of whole group instruction 
(Florida Center for Reading Research, 2006). 

Scholastic recommends that Read 180 be delivered 
to students in daily 90-minute instructional blocks. 
Using this model, students get 20 minutes of whole 
group instruction and three 20-minute rotations 
through the stations, followed by a 10 minute wrap 


up. Scholastic also suggests limiting enrollment in 
Read 180 classes to 15 students. While many school 
districts follow these recommendations closely, others 
do not have either the resources or flexibility to 
modify the school day or to drastically reduce class 
size to fit Scholastic’s recommendations. When this 
is the case, schools tend to mold the program to fit 
their specific circumstances. Some schools use the 
Read 180 program with average, or only slightly 
reduced class sizes. Others split the 90-minute 
instructional block into two 45 -minute periods within 
the same day, or even into two 45 -minute periods 
on consecutive days. The intervention has also been 
used as an after-school program, administered as 
infrequently as two times a week. 

Questioning the Author 

Questioning the Author is an instructional technique, 
rather than a complete literacy program or curriculum. 
It is designed to engage late elementary through 
high school students in critical reading, thinking 
and discussion. The goal is for students to improve 
comprehension and retention of the information 
presented in texts (Beck, McKeown, Hamilton, & 
Kucan, 1997). As such, it has been used primarily with 
content area texts, particularly in the social sciences, 
but the technique is intended to be appropriate for 
interactions with any type of text. Questioning the 
Author was developed by researchers at the University 
of Pittsburgh and Bethany College in cooperation with 
the Pittsburgh Public Schools. 

Using this approach to literacy instruction, teachers 
model their own reading processes for students (Beck, 
McKeown, Sandora, Kucan, & Worthy, 1996). In 
addition, they make use of a carefully constructed set 
of questions, referred to as queries. The queries are 
posed at planned intervals during the reading of the 
text and are designed to assist students in constructing 
meaning. Students are encouraged to see authors 
as fallible human beings who do not always express 
information and ideas clearly. Through student-to- 
student interaction, the group works collaboratively 
to demystify the text and uncover its more subtle 
meanings. Teachers use discussion strategies such 
as “marking” (drawing attention to an idea) and 
“revoicing” (using other words) to enhance student 
discussion and comprehension (Florida Center for 
Reading Research, 2006). 
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Questioning the Author requires few resources 
beyond professional development costs. However, 
the training time varies widely, ranging from 
approximately four to twelve days per teacher in the 
first year. Because teacher preparation is an integral 
part of this program, developers believe that new 
practitioners will need 1.5 hours to prepare for 
each lesson. This time demand is needed because 
appropriate queries must be planned out by teachers 



for each new text. However, most schools are unable 
or unwilling to give teachers the additional planning 
time needed for this preparation; as a result, teachers 
must either implement the technique without the 
suggested preparation time or spend many hours 
outside of school preparing texts and developing 
queries. The developers also recommend rearranging 
the classroom furniture into a U shape so that students 
can see each other’s faces during the discussion. Some 
teachers choose to follow this suggestion and others 
do not. 

Reading Apprenticeship 

Reading Apprenticeship is an approach to literacy 
that seeks to demystify academic reading for 
middle and high school students who struggle with 
text comprehension. Similar to Questioning the 
Author, it is not a complete curriculum so much as 
a pedagogical approach. In contrast to Questioning 
the Author, Reading Apprenticeship aims to root 


literacy instruction and practice in the subject areas 
by attending to the unique demands and practices 
within each discipline. Developed by the Strategic 
Literacy Initiative (SLI) at WestEd, it is based on the 
premise that remedial, basic-skills programs result 
in a “literacy ceiling” that can limit academic and 
other opportunities (Greenleaf, Schoenbach, Cziko, 

& Mueller, 2001, p. 86). To surpass these limitations, 
Reading Apprenticeship prepares educators from 
all content areas to embrace new and complex 
conceptions of reading as well as new ways to 
develop students’ academic reading skills. In Reading 
Apprenticeship classrooms, teachers and students act 
as partners in a collaborative inquiry into reading as 
they engage with texts in a specific subject area. 

In order to create classrooms where students 
function as active and effective readers and learners, 
Reading Apprenticeship trains teachers to reframe 
reading and writing in their subject areas by planning 
along four dimensions: social, personal, cognitive, 
and knowledge building. The social dimension 
focuses on creating and maintaining a supportive 
learning environment where students may feel 
comfortable making mistakes and asking questions. 
The personal dimension seeks to improve students’ 
identities and attitudes as readers. The cognitive 
dimension provides students with strategies and 
tools to aid comprehension, with an emphasis on 
group discussion of when and why certain tools are 
useful. The knowledge-building dimension involves 
recognition and expansion of the knowledge students 
bring to a text. These four dimensions are linked in 
the classroom through “metacognitive conversation,” 
a practice that makes the invisible aspects of each 
dimension visible and open for discussion (for detailed 
discussion of dimensions, see Schoenbach, Braunger, 
Greenleaf, & Litman, 2003). 

While it is possible for Reading Apprenticeship 
to be implemented by a single teacher, its developer, 
SLI, emphasizes the importance of cross-curricular 
implementation. In an ideal implementation, 
all teachers in a school will implement Reading 
Apprenticeship, meeting regularly to discuss progress 
and strategies. It is considered vital that “full 
implementation” schools make time for such meetings 
to occur. 

SLI trains educators in Reading Apprenticeship 
through a variety of professional development 
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opportunities, ranging from eight-day SLI series of 
sessions to one-day sessions provided by local “teacher 
experts.” Nevertheless, all professional development 
educators are trained to see reading differently 
through examining their own reading process, that 
of adult peers, and of students. Because Reading 
Apprenticeship focuses on “retraining” content-area 
teachers, program implementation does not require 
structural change to the school schedule, purchase of 
new equipment, or additional personnel. 

Data and Analysis 

Our project was constrained to three months during 
the summer of 2005, so time factors limited the more 
refined collection and analysis of data that would be 
required in a precise cost accounting. However, we 
believe that differences in the resource patterns among 
sites and the overall cost magnitudes are broadly 
representative. 

A five-step method was used to gather and organize 
data for this study; this approach is part of the 
“ingredients method” described in Levin & McEwan 
(2001, Chapter 3 & 4). 

First, we reviewed published program documents 
for each intervention. These documents included 
general program descriptions, implementation 
guidelines, reports by previous program evaluators, 
district and program websites, implementation 
videos, journal articles, and various other sources 
of information. This review of published materials 
familiarized us with the programs and alerted us to 
potential costs and pitfalls of implementation. 

Second, we contacted program developers by 
telephone and, where possible, met with them in 
person. Developers explained both the minimal and 
ideal resources required for successful implementation 
of their intervention. They described the primary 
obstacles to implementation as well as the resources 
and actions that schools and districts commonly 
used to overcome those obstacles. Each of the three 
program developers provided a wide-range of program 
literature and contact information for successful 
program implementers around the country. 

Third, we conducted telephone interviews with 
school and district personnel — teachers, principals, 
technology specialists, district literacy coordinators, 
and superintendents — to learn how programs were 
being implemented at the local level. Our goal was 


to document and understand the various ways that a 
single model took on different operational features 
at the sites of implementation. Respondents at each 
site were asked not only to describe the pattern of 
implementation, but also to identify the characteristics 
of personnel required for successful program 
implementation. For example, one district noted the 
necessity of a full-time district literacy coordinator, 
while another had no such position. In addition, 
respondents described the nature of the professional 
development offered to teachers, administrators, and 
technicians. Finally, they described the materials and 
the facilities required to implement the program. 

Some of the interventions required the purchase 
of additional technology and the procurement of 
additional classrooms while others did not. At each 
of these stages, local staff described problems they 
encountered as well as the resources and actions they 
used (or tried to use) to overcome these problems. 

The short timeframe of this study (June to August 
2005) and the lack of program implementation over 
the summer were limiting factors in this phase of 
the study. In some cases, it was difficult to schedule 
interviews because school personnel were out of 
the office for the summer. We note the number of 
sites contacted at the beginning of each program’s 
results section. Our limited time frame also made it 
impractical to conduct observations on site. 

Fourth, we used the above sources of information 
to construct an ingredients list for each site’s 
implementation. This list outlined the personnel, 
materials and facilities used at each implementation 
site, as compared to the developers’ recommended 
ingredients. The purpose of this method was not 
to highlight inconsistencies between implementer 
approaches and developer models, but rather to show 
future schools and districts the real range of resources 
required to implement a given intervention. By 
identifying the ingredients that are actually used in an 
intervention, we hope to inspire schools to think more 
deeply about the resources, time requirements, and 
personnel needs that contribute to program success. 

Fifth, we created Excel spreadsheets detailing the 
ingredients and relative costs for each intervention 
across different sites. Costs were assigned to the 
ingredients using national averages, developer 
cost estimates, and individual site estimates. Total 
program costs were determined as well as program 
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costs per student. Although major cost components 
such as program licenses, professional development, 
and computers are purchased in the first year, they 
continue to provide services over a number of years; 
using proper costing techniques, these costs were 
annualized where appropriate. 2 That is, only that 
portion of the cost of such ingredients that should 
be charged to a single year of use is included in these 
estimates. To annualize costs, we assumed five years 
of program implementation at a 5 percent discount 
rate. Five years is also a number recommended by 
Scholastic for Read 1 80, although the program has 
been implemented beyond five years in some sites. 
Exceptions to the five year expected lifespan are noted 
in the tables. 

Findings 

The following presentation describes the findings 
from analysis of site implementation of each of the 
reforms. Since this work was performed over the 
summer, we were limited in both time and access 
to school personnel in obtaining data. Accordingly, 
what follows should be viewed not for its precise 
cost analysis as much as for its patterns of resource 
use and the magnitudes of cost. Also, our purpose 
is not to compare the cost of different intervention 
models, because some are more modest than others, 
a factor that may be reflected in both their scope 
and effectiveness, and because they have distinctively 
different goals. Results for each program are 
summarized separately below and in Tables 1-3. Each 
of the tables report for one program the costs of that 
program’s ingredients, the costs at an “idealized” site 
described by the developers, and the costs at one 
or more actual implementation sites. We close by 
offering our readers some conclusions and insights 
gained through this study. 

Read 180: Ingredients and Costs 
The list of ingredients for Read 180 was derived 
from extensive telephone conversations and emails 
with numerous sites suggested by Scholastic, the 
sponsor of Read 180. Three of these sites, chosen 
for their diversity in geographic region and school 
size, are included in Table 1. In addition, we 
obtained details on the intervention from meetings, 
phone conversations, and correspondence with 
Scholastic representatives. The main categories of 


cost ingredients include personnel, professional 
development, facilities, equipment and materials, 
and licenses purchased. Table 1 provides a listing of 
ingredients with the additional quantities of each for 
three Read 180 school sites and the recommended 
model of Scholastic. 

Personnel costs were divided into five categories: 
school administrators, school technicians, district 
coordinators, district technicians, and additional 
teachers required for program implementation. 
Additional teachers are required for Read 180 when 
schools cut class sizes for the program and/or alter 
their schedules to accommodate the recommended 
90 minute class period. Therefore, we calculated 
the number of additional teachers needed for Read 
180 where the Read 180 requirements deviated from 
existing class sizes and period lengths. 

Clearly, the simple purchase of Read 180 
courseware in itself is inadequate to ensure increased 
student literacy achievement without appropriate 
staffing, professional development, and use of the 
courseware. One purpose of this report is to make 
the less conspicuous costs of adolescent literacy 
programs and their appropriateness visible to future 
implementers such as non-teacher personnel. All 
sites reported that district leadership and support 
are required to initiate and sustain an effective 
implementation of Read 180. For example, a teacher 
from a large urban district reported that, in its first 
four years of Read 180 instruction, his school had four 
different principals, none of whom were committed 
to Read 180. As a result, there was vast inconsistency 
in implementation, with children constantly being 
shifted in and out of Read 1 80 classes. In the fifth year 
of implementation, the school hired a principal who 
was supportive of Read 180 and, for the first time, 
the teacher had the same students from September 
to June. In addition, support for program challenges 
was readily available, as were resources for program 
essentials such as headphones and technical support. 

In this case, both additional principal time and school 
resources were needed to maximize the success of the 
program. The above description of essential support 
systems was echoed by Scholastic, as well as teachers, 
principals and district personnel at all sites. 

While some districts reported few technology 
problems, others described technology as a primary 
obstacle to program implementation. In all cases, 


8 I 


TIME TO ACT 


technology-related personnel provided essential 
support to Read 180 teachers. Many schools employ 
on-site technology specialists to resolve problems 
quickly, and all districts in our sample use a district- 
level technology expert who travels from school to 
school, resolving hardware and software problems. 

To determine program costs for school 
administrators, school technicians, district 
coordinators, and district technicians, we converted 
the amount of time spent on Read 180 per school- 
year into a percent of a position (assuming 1440 
work hours per year) and divided that number by the 
average national salary for that position. All personnel 
costs were calculated using national averages for the 
2004-2005 school year (Educational Research Service, 
2005), plus 25 percent estimated fringe benefits. 

Scholastic provides numerous options for Read 180 
professional development, some of which are included 
in the cost of the program licensing. According to 
Scholastic, two days of implementation training for 
teachers as well as the teachers’ participation in its 
online course — both provided with the purchase 
of Read 180 — are necessary in the first year of 
implementation. Optimally, Scholastic recommends 
that districts purchase a selection of additional half- 


day seminars and/or additional online reading courses. 
For school and district level administrators, Scholastic 
recommends participation in a half-day leadership 
development course, included in the price of the 
program. Finally, for technicians, Scholastic provides a 
Read 1 80 Technical Training Program at an additional 
cost of $9,000 for one day or $12,000 for two days. 
The training prepares technicians to provide program 
support within their school environment. 

Additional professional development costs that are 
not included with the purchase of Read 180, but are 


important for schools to consider, include substitute 
costs (where required), additional teacher training, and 
the opportunity costs associated with time spent on 
Read 1 80 training. Because professional development 
is intended to exert an impact beyond the year that it 
is provided, the costs are annualized (i.e., divided into 
yearly costs). 

Additional classrooms make up the primary facility 
requirement for Read 180 to accommodate reduced class 
size. 3 While bcholastic provides no recommendation 
for facilities, it is important to note that Read 1 80 
classrooms must be large enough to house computers 
for one third of the students as well as provide 
sufficient space for the small groups, independent 
reading, and computerized instruction groups. 

Because Read 1 80 is a technology-based 
intervention, equipment and materials are vital to 
program implementation. Student computers and 
application servers comprise the largest equipment 
cost. Flowever, this cost varies depending on the 
existing technological infrastructure of a school. 

For example, a school without adequate models or 
numbers of computers for Read 180 will incur greater 
first year technology costs than will a school with 
the proper infrastructure already in place. Scholastic 

recommends a specific 
arrangement of system 
requirements for the best 
performance of Read 180; 
however, it acknowledges 
local differences in 
technology infrastructure 
and makes recommendations 
to schools and districts based 
on the technology that they 
already have. 

While computers and 
servers comprise the largest equipment costs, full 
implementation also requires a printer, headphones, 
and cassette/CD players. Districts can also purchase 
additional classroom books and project reading kits. 
Costs for equipment and materials were annualized 
based on their average lifespan. 

In order to determine the costs of implementing 
$cholastic’s recommendations for Read 1 80, one 
must know the situation of the school prior to 
implementation. Much of the cost depends on pre- 
existing local conditions such as class size, technology 
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infrastructure, length of class periods, and personnel 
characteristics. Using information from three districts, 
all of which enthusiastically endorse Read 180, we 
were able to calculate the range of approximate costs 
associated with implementing this program. These 
results are found in Table 1 . 

READ 180 SITE ONE 

Site One is a large urban/suburban school district with 
an enrollment of close to 300,000 students. During 
the 2004-2005 school year, this district used the Read 
180 program to instruct approximately 6,700 students. 
Initially, Site One adopted an “early bird” schedule in 
which students arrived at school prior to the beginning 
of the regular school day. Attendance was a serious 
problem, so Site One modified its schedule to allow 
students to receive Read 1 80 instruction daily, in 90- 
minute blocks during the school day. In addition, Read 
180 classes are capped at 30 students, a significant 
reduction from the 38 student per class district average 
reported by the district. 4 However, since we could not 
confirm that the average class size was initially this 
large, we have used a class size of 30 as the initial level. 
Bear in mind that Scholastic recommends a maximum 
class size of 15 for Read 180. 

At Site One, the Coordinator of Instructional 
Programs coordinates Read 180. She spends 
approximately 90 percent of her time on Read 
180-related activities, which includes meeting with 
Scholastic and district personnel, observing teachers 
and reporting Read 180 results to interested parties. 
Each of the 81 schools that are using Read 180 has 
its own Educational Computer Strategist (ECS), who 
spends approximately one hour per week on Read 1 80, 
usually resolving problems with computer hardware. A 
technical field manager trains the ECSs and provides 
specialized knowledge on the Read 1 80 software as 
needed. All of the district’s 210 Read 180 teachers 
attended one-day of professional development prior to 
implementing the program in their classrooms. 

Site One purchased 185 stages of 60 Read 180 
licenses in 2004, giving it the capacity to serve 11,110 
students with the intervention; however, during the 
last school year only 6,701 students received Read 
180 instruction. The reasons for this underutilization 
varied from school to school and included lack of 
administrator support, the inability of teachers to 
manage the small group structure of a Read 1 80 


classroom, and a lack of school funds for Read 180 
materials. These implementation problems and 
the resulting idle licenses greatly increased the per 
student cost of Read 180 for Site One. An additional 
37 teachers are needed to accommodate the time 
requirement for Read 180 extended class periods. 5 The 
salaries and benefits for these teachers constituted the 
other major expense in implementing Read 180 at Site 
One. The cost per student at Site One for 2004-2005 
was estimated to be about $600 per student. 

READ 180 SITE TWO 

Site Two is significantly smaller than Site One, serving 
an enrollment of almost 48,000 students, with 1,080 
in Read 180 classrooms during the 2004-2005 school 
year. Site Tvo adheres closely to the Scholastic 
model. Read 1 80 classes are limited to 1 5 students, 
half the size of the reported, average middle school 
language arts class. In addition, class periods for 
Read 180 are 90 minutes long, which is twice the 
average class period length in the district. With these 
two modifications the school district would need to 
hire approximately 18 additional teachers, without 
reducing other school programs. The additional 
personnel cost is by far the largest resource burden 
for Site Two. 

In this district, the Secondary Reading Supervisor 
is responsible for overseeing Read 180. Managing 
the program occupies about one third of her time. 
While Scholastic provides one day of training to 
teachers, the Secondary Reading Supervisor provides 
seven additional days of training to Read 180 teachers 
throughout the school year. This extra professional 
development necessitates substitutes for the 5 8 
teachers who use Read 180. 

Four district-level computer technicians work 
exclusively on Read 180. 6 Their job entails providing 
hardware and software support to schools, updating 
computer programs, and running the district’s 
unique centralized computer system, which enables 
the district coordinator to see and manage student 
data from the district office. Computer technicians 
stationed at each school deal with simple hardware 
problems related to Read 180 in addition to non-Read 
180 technology issues at the school. 

In addition to the classroom stations suggested by 
Scholastic, this school district has a “computers down” 
station in each classroom. This area contains skills 
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cards, and other non-computerized reading activities, 
and allows teachers to continue using the Read 180 
small group instructional model even when the 
computers are not working. 

The cost of implementing Read 180 at Site 
Two is about $1,500 per student, the highest in our 
study. Reducing class size by 50 percent for Read 
180, doubling the instructional periods and hiring 
four district technology experts, contributes heavily 
to this cost. Additionally, Site Two’s higher costs 
may be attributable to the relatively small size of its 
implementation and the attempt to centralize the data. 

READ 180 SITE THREE 

This school district is a suburban district that enrolled 
about 42,000 students in 2004-2005. Read 180 is 
used to remediate literacy instruction for about 
2,400 students. As was true for Sites One and Two, 

Site Three substantially reduces class size in Read 
180 classes from a reported average of 38 to 24 
students per class. However, because we could not 
confirm officially the initial class size, we calculate 
the costs based upon an average class size of 30. 

Unlike the other sites, Site Three does not modify 
the school schedule for Read 180, so students receive 
45-55 minutes of instruction daily instead of the 
recommended 90 minutes. 

The implementation of Read 1 80 at Site Three is 
facilitated by the Program Specialist for Literacy in 
Secondary Education. She spends about 80 percent 
of her time overseeing Read 180. Part of her job is 
augmenting the two-day implementation training 
offered by Scholastic with 2-4 additional training 
days for teachers. All of the training takes place during 
the school year, so substitute teachers are hired to 
cover the Read 180 classes. Read 180 teachers are 
also asked to participate in monthly meetings outside 
of their contract time, for which they are paid an 
hourly wage. 

School level “micro-computer specialists,” 
employed by most high schools and some middle 
schools spend about 2 hours a week per Read 180 
classroom. They perform routine maintenance on 
Read 180 computers and programs. When schools 
cannot afford a micro-computer specialist the teachers 
and district technician spend more time on the 
technological aspects of the program. One district 
level Read 180 technician works with all of the schools 


and trains the micro-computer specialists, (initially, 
the district purchased a two day technology training 
from Scholastic). Read 180 maintenance, upgrades and 
trainings occupy about 40 percent of his time. 

Currently, Site Three spends about $285 per 
Read 180 student. The cost is significantly lower 
than those of the other sites because this district uses 
45-55 minute periods as opposed to the 90 minute 
suggested class periods. While we cannot comment on 
the effectiveness of this approach, the students at Site 
Three receive half as much Read 1 80 instruction as 
those at the other two sites, allowing the teachers to 
instruct twice as many students. 

READ 180 SUMMARY 

Table 1 compares the ingredients and costs of 
implementation at the three Read 1 80 sites and 
for the Scholastic recommended model. Bear in 
mind that the overall numbers are sensitive to the 
scale of implementation, but the per-student cost 
provides a reasonable picture of the difference in 
magnitude of the costs at each site and the Scholastic 
recommendation. There is remarkable variability 
in implementation logistics and the consequent 
differences in costs. For example, the recommended 
Scholastic model for implementation would entail 
about $1,100 in costs if followed faithfully in a 
district with a class size of 30. 7 But, in Site Three 
the cost is only one-quarter of this amount because 
there was only a small reduction in class size (30 to 
24 students) and no increase in instructional time. 

This comparison also illustrates the substantial 
impact on costs of changes in class size and length of 
instructional period. For example, the costs imputed 
for the additional teacher resources in the Scholastic 
Model are about $950 a student, far in excess of the 
relatively modest charges for licensing the program 
and the equipment that is required. Clearly when 
class size is kept constant or reduced only slightly 
or instructional time is maintained or increased 
only a small amount, the costs of Read 180 are also 
reduced. However, these deviations from Scholastic’s 
recommended implementation model might seriously 
impair effectiveness. 

Questioning the Author: Costs and Ingredients 

The ingredients needed to successfully implement 
Questioning the Author (QtA) were determined 
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through extensive oral and written contact with the 
developers at the University of Pittsburgh, and with 
one implementation site. Although we contacted other 
sites repeatedly, we were unable to conduct interviews 
due to the timeframe of this study. 

QtA is a professional development program that 
aims to equip teachers with new tools for engaging 
students in text and curriculum. Therefore, it does not 
require additional materials or modifications to the 
school day. Because there are very broad guidelines for 
class size and no recommendations for period length, 
it is extremely unlikely that schools will hire additional 
staff for the express purpose of implementing QtA; 
however, it is suggested by the developers that a 
minimum of two teachers per school are prepared 
so that they can plan lessons and provide support to 
each other. 

Costing results for QtA are found in Table 2 . 

The main costs of this program are incurred for 
professional development. In addition to the materials 
for teachers and the initial training, schools are 
expected to hire consultants to observe each teacher 
eight times during their first year of implementing 
QtA. Each observation costs $187.50, meaning that 
the observation bill for one teacher is $1500. Like 
other one-time professional development expenses, 
this cost is annualized over the expected life of the 
program, making it about $350 per year. 8 Alternatively, 
districts can hire a full time facilitator to train teachers 
and do the observations. The developers estimate that 
someone in this position would need to spend about 
three hours per month with each teacher-implementer. 

The developers suggest that administrators attend 
the one-day training, and do about three observations 
of each teacher so that they understand and are 
able to support the teacher-implementers. Both of 
these activities have opportunity costs, which are 
calculated by multiplying the administrator’s salary 
and benefits by the percentage of time that person 
spends on the program. Using national averages for 
administrator salaries, we calculated this cost to be 
about $1,300 annually. 

A hypothetical high school or middle school that 
trained two teachers in QtA following the developer’s 
guidelines would spend only a very modest amount of 
about $11.00 per student annually. The emphasis is 
on incorporating in the curriculum and the teachers’ 
repertoires the capacity to ask questions in a manner 


that elicits thinking and articulate responses from 
students. The very low cost reflects the fact that no 
modifications are required in class size, scheduling, 
personnel augmentation, or additional facilities 
and materials. 

QUESTIONING THE AUTHOR SITE ONE 

$ite One is a district with a total enrollment of 3,200 
students. In this district, Questioning the Author has 
been implemented in language arts classes in grades 3 
through 8. Because most of the teachers who adopted 
the approach work in elementary schools, they only 
teach one group of students per day. 

In addition to a two-day initial training provided by 
the developers, 25 teachers and three administrators 
received QtA-related professional development one 
day per month throughout the year. Removing the 
teachers from classrooms for nine days incurred 
$27,000 in substitute teacher costs. 9 The other large 
cost for QtA $ite One was for teacher observations. 

At eight observations per teacher, the district paid an 
estimated $37,500 to the University of Pittsburgh for 
consultants to do observations. Annualized, this cost 
becomes $8662.50 per year over a five year period. 

The cost of implementing QtA at $ite One is 
estimated to be about $35 per student per year. This 
cost estimate may be low because teacher-training 
time was not included in the analysis. The higher per- 
student cost at QtA $ite One is attributable to a one- 
day a month professional development session for all 
teachers which is not required by the developer of the 
model. Still, the cost per student is very modest. 

An interesting note is that this district reported 
very little teacher turnover. The director of 
Elementary Curriculum and Instruction described it 
as a place where “teachers get a job and stay for their 
career.” From this perspective, it makes sense to invest 
heavily in professional development because teachers 
may use the technique to benefit students in that 
district long after the professional development period 
is over: clearly an important consideration for model 
choice and implementation. 

Reading Apprenticeship Ingredients and Costs 

The list of ingredients for Reading Apprenticeship 
(RA) was obtained through reviews of program 
literature and from telephone conversations with 
the developer, $LI at WestEd and implementers 
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from multiple sites, two of which are represented 
in Table 5.3. The primary ingredient categories for 
RA are personnel and professional development. 
Because RA is a professional development process 
that trains teachers to think and teach in a new way, 
there are no facilities or equipment costs associated 
with implementation. 

Schools and school districts need not hire 
additional teachers to implement RA because the 
program is delivered by content-area teachers in their 
content-area classes. While the personnel costs for 
teachers do not change with RA, the program does 
incur opportunity costs for school and district level 
administrators time. To determine these costs, we 
converted the amount of time spent on RA per 
school year into a percent of a full-time position and 
divide that number by the average national salary for 
that position. 

Rather than endorse a specific model of 
implementation, SLI provides schools and school 
districts with a range of professional development 
options to choose from. The National Institute in 
Reading Apprenticeship (NIRA) is an eight-day 
“training-of-trainers” program designed to prepare 
school, district, or department leaders to train 
teachers in local professional development sessions 
or implement RA in their own classrooms. Site- 
based trainings, provided by SLI staff or certified 
RA consultants, range in length from one to seven 
days. Finally, the Bay Area Network Series is a seven- 
day program, similar to NIRA but designed for 
educators in the Bay Area. 10 Following the first year 
of implementation, SLI offers continued training 
that is included with the purchase of the program. 
Other significant costs associated with professional 
development include substitute costs and opportunity 
costs for school and administrator training time. 

In evaluating the data below, it is essential to 
remember that both sites from which we obtained 
information were recommended by SLI, and both 
enthusiastically endorsed RA. Thus, it is possible 
that our data: (1) do not account for the full range of 
variety in RA implementation and (2) do not account 
for implementation obstacles experienced by less 
successful districts. Despite the fact that both sites 
have experienced success with RA, we found a wide 
difference in implementation between the two sites 
which had a large impact on costs. Because this is not 


a cost-effective analysis, however, we cannot comment 
on the relative effectiveness of the two approaches. 

READING APPRENTICESHIP SITES ONE AND TWO 

Reading Apprenticeship Site One and Site Tvo are 
both rural districts in the same state. During the 
2004-2005 school year, Site One, a district comprised 
of five high schools, trained 42 teachers and served 
approximately 1,270 students. In the same year Site 
Tvo, implementing RA in one large high school, 
trained 57 teachers and served approximately 1,150 
students. While both sites trained similar numbers of 
teachers who served similar numbers of students, the 
costs per student were significantly different with Site 
One spending just over three times more per student 
annually. These differences are due to different 
methods of implementation, but the overall cost at 
both sites is relatively modest because no additional 
personnel, materials, or facilities are needed. 

The primary costs for Site One were in 
professional development. The district customized 
the site-based training to provide 42 teachers with 
five days of training by SLI experts for $60,000. In 
addition, it sent 8 teacher-leaders to participate in 
NIRA for the cost of $33,000. These teachers returned 
to their schools as leaders of program implementation. 
The total substitute costs for these trainings were 
around $28,000." 

While Site One paid SLI around $90,000 for 
professional development, Site Tvo paid only $4,000 
because it trained its teachers “in house.” Site Tvo 
sent four teacher-leaders (including one administrator) 
to the Bay Area Network Series and these teachers, 
rather than SLI experts, provided training to the site’s 
53 remaining teachers during monthly professional 
development meetings, one third of which were set 
aside by the principal for exclusive focus on RA. 12 The 
substitute costs for the Bay Area trainings were around 
$3, 360. 13 It is important to remember that in both 
models the professional development costs are low 
estimates because we do not account for initial teacher 
training time. 

Outside of professional development, the 
primary cost to both sites was for school and district 
administrator time. While the costs do not seem high, 
it is essential that this category not be overlooked 
by future implementers. In $ite One, the assistant 
superintendent spent approximately 70 hours per year 
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on RA while in Site Two the assistant superintendent 
spent about 1 00 hours per year, clearly significant 
numbers considering the numerous responsibilities and 
obligations of top district administrators. In both sites, 
this time was spent in staff training meetings, working 
with program developers and school administrators, 
securing program resources, organizing logistics, 
etc. While such time commitments are clearly not 
“officially” required for program implementation, 14 
both sites emphasized that without such strong district 
involvement and organization, it would be difficult to 
consistently implement the intervention at a high level. 

In addition to the large time investment by district 
administrators, there are three other similarities that 
are important to note. First, school administrators at 
both sites attended the vast majority of teacher-related 
professional development sessions. At Site Two, for 
example, the three top administrators attended the Bay 
Area Network training over the course of three years, 
along with most in-school collaborative meetings and 
trainings. Second, both sites adhere closely to SLI’s 
recommendation that RA be “embedded in subject- 
area instruction across the curriculum, rather than 
becoming the sole purview of the English department” 
(Greenleaf, Schoenbach, Cziko, & Mueller, 2001, 
p. 89). Taken together, these two factors advanced 
implementation by creating collaborative cultures of 
literacy with extensive administrative support. 

Third, both sites reported few problems with 
teacher turnover. Because RA is an approach to 
the professional development of teachers, the risks 
associated with turnover are high. For example, 
a teacher who is trained in his second year and 
leaves by his fifth will raise the annualized cost of 
RA implementation by reducing its lifespan. Thus, 
local retention rates should be considered when 
generalizing RA data across districts. That being 
said, high turnover schools can minimize this risk by 
selecting the teacher-leaders who are most likely to 
remain at the school over time. 

Recommendations 

for Successful Implementation 

An important finding from this study is that 
implementation costs may vary considerably 
from setting to setting because of differences in 
implementation. Some of the variation in costs may be 
due to different prices for resources among areas, such 


as differences in teacher salaries and benefits among 
places with low costs of living and high costs of living. 
These are not reflected in our data because we used 
an average of “national” prices in estimating the costs. 
Other factors could cause differences in the cost of 
implementing the same program, such as the presence 
of students with special needs (immigrants and English 
language learners), or the fact that the school happens 
to be located in a poor area. However, we believe that 
most of the difference is simply due to differences in 
implementation among school sites, with some using 
more resources for the same intervention. 



We believe that if schools were to put into practice 
the following recommendations, they could provide 
more effective implementation and better monitoring 
of costs. 

1 . Selection of Intervention — When selecting 
a reform or intervention schools should set aside 
sufficient time to gather appropriate information 
and to include discussions and input by teachers and 
other staff who will be involved in implementation. 
Considerable experience affirms that staff agreement 
on goals and knowledge of and commitment to 
reforms provides greater promise of success. Datnow 
(2000) has emphasized that the process of participation 
of teachers and other staff in becoming informed 
about the issues leading to new interventions and the 
choice of interventions is key to their cooperation. 
However, she found that often this process of school 
“buy-in” has been carried out in a perfunctory manner, 
culminating in a ritual vote that reaffirms the obvious 
and declared preference of key administrators. An 
authentic process of informing staff and obtaining 
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their input is more likely to enhance school-wide 
understanding of the need for change and instill a 
widespread willingness on the part of staff to get 
involved in both the choice of an intervention and its 
effective implementation. 

Given the presence of many reforms dedicated to 
improving adolescent literacy, it is crucial to attempt 
to match potential choices of reforms to student 
needs and the capacity of a school to implement the 
intervention. A review of the various alternatives will 
reveal that different reforms have been developed for 
different groups of students (e.g., English language 
learners) or students with different learning needs. 
Evaluations of results will also emphasize where these 
reforms have shown success. Further, the descriptions 
of the interventions will suggest strengths that schools 
might build on such as experience with the use of 
educational technologies or particular approaches to 
student grouping or scheduling that match up well 
with specific reforms. Obviously, it is better to choose 
potential reforms which match school strengths than 
to require the schools to develop major new ones in 
order for a reform to succeed. 

2. Implementation Requirements — The adoption 
of a reform that matches the needs and strengths 
of a school is based upon the overall features of the 
school and reform. However, such a decision does not 
address the details of implementation and their costs. 
The ingredients method allows for both planning the 
intervention and ascertaining its costs. This method 
calls for decision-makers to identify and specify all of 
the resources and conditions that will be required to 
make the reform a success. Details on identification of 
ingredients, personnel, facilities, equipment, supplies, 
and other resources can be found in Levin and 
McEwan (2001). The goal of this activity is to be clear 
about what will be needed with special attention to 
the qualitative dimensions of the resources such as the 
qualifications of appropriate personnel. Information 
on required ingredients can be obtained from three 
sources. First, developers can provide descriptive 
materials and information as well as referring to other 
sites that have adopted the reform. Second, these 
other sites can be contacted, and visited if close-by, to 
observe the program. Third, practitioners at these sites 
can be interviewed on details of their implementation, 
as well as lessons learned from their experience. All 
of this information can be integrated into a plan for 


implementation and the resources necessary for the 
plan to succeed. 

3. Costs and Resource Availability — Tvo key 

questions on which good implementation rests are: 
“Has adequate funding been put aside to cover the 
cost of the reform?” And, “Are the appropriate 
resources available?” The way to ascertain the 
answer to the first question is to know the cost of 
the reform by placing a cost on all of the ingredients 
(Levin & McEwan, 2001). Not all of the ingredients 
require additional funding if some of them can be 
obtained through reallocation of existing resources 
from less productive uses. Many reforms stumble 
because available personnel in the school such as 
administrators, coordinators, and coaches do not have 
the skills or experiences that are necessary to provide 
support for the reform. This may place the school in a 
difficult situation where personnel must be marshaled 
from those internally available in the school or district, 
but those who are readily available are inappropriate. 

From the analyses of cost and resource availability, 
it is possible to ascertain both the obstacles to 
implementation success as well as possible solutions 
for overcoming those obstacles. If the costs exceed 
the resources that are available, it is important to seek 
additional resources or to decide how to accommodate 
reform within available resources. For example, the 
cost of additional personnel and space for reducing 
class size to some prescribed level may exceed the 
funding and space that are available. Schools will 
need to confer with developers on how to address 
this shortcoming or if successful implementation can 
take place despite this shortfall. A similar analysis 
must be done in terms of assuring that appropriate 
personnel are in place. At the point of implementation 
planning, the specific personnel who will work with 
the reform should be noted. This is important for 
two reasons. First, the particular personnel should be 
familiarizing themselves with the reform and their 
roles, well in advance of the actual implementation. 
Second, those who are planning the implementation 
need to size up required qualifications with those of 
the potential appointees. If available personnel are 
inappropriate, the organizers of the reform will need 
to seek alternatives or confer with the developers on 
what might be done. 

4. Implementation Plan — All resource 
requirements are identified, and provision is made 
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for their availability for the reform. Funding is 
adequate to cover their costs or to provide appropriate 
resources from within the school organization. 

Plans are made to acquire materials, software, and 
equipment in sufficient time to launch the reform. A 
timeline and calendar for professional development 
and other activities such as monitoring, classroom 
observations, feedback, and evaluation of results 
must be set out. If the school site lacks the funding 
and available resources to implement the reform 
along the lines recommended by the developer, 
discrepancies will become obvious and there will be 
time to enable a search for alternatives. Minimally, 
this approach to costing and implementation planning 
will establish whether the reform is feasible in the 
sense of the school having the operational and 
financial capacity to undertake it. More promising 
is the possibility that such planning will provide a 
blueprint for implementation, avoiding many of the 
unpleasant surprises and unintentional compromises 
that many schools have had to face in the process of 
implementing reform. 

Conclusion 

Although the developers of the most up-to-date 
adolescent literacy reforms have made considerable 
investments in constructing and testing their models, 
these efforts do not guarantee that the intervention 
will produce results when adopted in a specific 
school setting. Adoption of a promising reform, by 
itself, is not sufficient to assure that the reform has 
predictable costs and effectiveness. How a given reform 
is implemented determines its probability of success 
or failure, as well as its overall cost. Careful planning 
and analysis prior to launching a reform will likely 
provide both better implementation and better cost 
management. Such planning offers schools and districts 
ways to break out of the endless cycle of hope, effort, 
and disappointment that unfortunately has afflicted so 
many attempts at improving students’ literacy. 
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TABLE n,l 1 . Read 180 Implementation Models 


INGREDIENTS LIST 

INGREDIENT 

SCHOLASTIC 


Costs 

Annual 

Inputs 1 

Annual 


Costs 

Costs 


LICENSES (Packs of 60) 

$32,000.00 

$7,392.00 2 

1 

$7,392.00 

PERSONNEL (FTE) 

Additional teachers 3 

$57,355.00 4 

$57,355.00 

1 

$57,125.58 

Read 180 teacher: student ratio 



1:15 


Percent reduction from 30 student class size 5 



50% 


School level administration 

$105,282.50 

$105,282.50 

no rec. 6 


In-school technician 

$57,355.00° 

$57,355.00 



Read 180 district technician 

$95,385.00 

$95,385.00 



Read 180 district coordinator 

$85,892.50 

$85,892.50 

no rec. 


PROFESSIONAL DEVELOPMENT 

1st year teachers trained 10 



i 


1st day implementation training 

included 


yes 


2nd day implementation training 

included 


yes 


Red online course 

2 included 


yes 


Seminar series 

$2,500.00 

$577.50 

2 optimal 


Independent district training 





Non-contract training time 





Substitute days 

$120.00 

$27.72 



Total teacher training days 



2 


Administrator training time 

$105,282.50 

$24,320.26 

0.003 

$315.85“ 

Technician training time 

$57,355.00 

$13,249.01 

.005 

$66.25 

Half day implementation training 

included 




1 day implementation training 

included 


Yes 


1 day technical training 

$9,000.00 

$2,079.00 



2 day technical training 

$12,000.00 

$2,772.00 




FACILITIES 


no rec. 


Classrooms 






TABLE n °- 1 . Read 180 Implementation Models (continued) 


INGREDIENTS LIST 

READ 180 SITE ONE 

READ 180 SITE TWO 

READ 180 SITE THREE 


Inputs 

Annual 

Inputs 

Annual 

Inputs 

Annual 


Costs 

Costs 

Costs 

LICENSES (Packs of 60) 

186 

$1,372,472.64 

18 

$133,056.00 

40 

$295,680.00 

PERSONNEL (FTE) 

Additional teachers 3 

36.86 

$2,114,105.30 

17.93 

$1,028,375.15 

3.12 

$178,947.60 

Read 180 teacher: student ratio 

1:30 


1:15 


1:24 


Percent reduction from 30 student class size 5 

0% 


50% 


20% 


School level administration 

n.r 7 


n.r. 


0.02 8 

$2,105.65 

In-school technician 

2.36 

$135,357.80 

.45 

$25,809.75 

0.83 

$47,604.65 

Read 180 district technician 

1.125 

$107,308.13 

4 

$381,540.00 

0.4 

$38,154.00 

Read 180 district coordinator 

0.9 

$77,303.25 

0.33 

$28,344.53 

0.8 

$68,714.00 

PROFESSIONAL DEVELOPMENT 

1st year teachers trained 10 

270 


58 


37 


1st day implementation training 

yes 


yes 


yes 


2nd day implementation training 

no 


no 


yes 


Red online course 

limited 


no 


no 


Seminar series 

no 


no 


no 


Independent district training 



7 days 


3 days 


Non-contract training time 





9 hrs. 

$5,994.00 

Substitute days 

270 

$7,484.40 

348 

$9,646.56 

185 

$5,128.20 

Total teacher training days 

270 


464 


185 


Administrator training time 

0.003 

$72.96 

0 

$0.00 

0.001 

$33.78 

Technician training time 

0.243 

$3,219.51 

0.132 

$1,748.87 

0.07 

$883.18 

Half day implementation training 

81 


no 


no 


1 day implementation training 

no 


no 


no 


1 day technical training 

no 


22 

$2,079.00 

no 


2 day technical training 

no 


no 


1 

$2,772.00 

FACILITIES 

Classrooms 

166 


18 12 


20 
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TABLE v». I , Read 180 Implementation Models (continued) 

INGREDIENTS LIST 

INGREDIENT 

SCHOLASTIC 



Costs 

Annual 

Costs 

Inputs 1 

Annual 

Costs 


EQUIPMENT AND MATERIALS 

Student computers 

$600.00 

$138.60 

5 

$693.00 

Application server 

$2,000.00 

$462.00 

1 

$462.00 

Printers 

$120.00 

$27.72 

1 

$27.72 

Cassette players 

$4.00 

$4.00 

5 

$20.00 

Headphones 

$10.00 

$10.00 

5 

$50.00 

Additional books for classroom 

$499.00 

$183. 23 13 



Project achievement reading kits 

$169.00 

$62. 06 14 



Total cost 




$66,152.39 

Students served 




60 

Cost per student 




$1,102.54 
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TABLE V"- 1 . Read 180 Implementation Models (continued) 


INGREDIENTS LIST 

READ 180 SITE ONE 

READ 180 SITE TWO 

READ 180 SITE THREE 

Inputs 

Annual 

Costs 

Inputs 

Annual 

Costs 

Inputs 

Annual 

Costs 

EQUIPMENT AND MATERIALS 

Student computers 

1660 

$230,076.00 

108 

$14,968.80 

144 

$19,958.40 

Application server 

85 

$39,270.00 

18 

$8,316.00 

20 

$9,240.00 

Printers 

166 

$4,601.52 

18 

$498.96 

20 

$554.40 

Cassette players 

n.r. 


n.r 


144 

$576.00 

Headphones 

n.r. 


n.r. 


144 

$1,440.00 

Additional books for classroom 

n.r. 


yes 


18 

$3,298.19 

Project achievement reading kits 

n.r. 


18 

$1,117.02 

50 

$3,102.84 

Total cost 


$4,019,271.50 


$1,635,500.64 


$684,186.89 

Students served 


6,701 


1,080 


2,400 

Cost per student 


$610.55 


$1,514.35 


$285.08 


1 Reported by Scholastic Read 180 National Implementation Manager. 

2 All one time costs are annualized over five years using a five percent discount rate unless otherwise noted. 

3 Additional teacher formula, assuming a six period day and class size of 30: 100/(Read 180 Class Size * Read 180 periods per day)-X ; 100/(30 *6 periods per 
day)-. 56 ; (X-.56)/ 100— new teachers per Read 180 student * number of Read 180 students severed - number of additional Read 180 teachers. 

4 All personnel costs were calculated using national averages for the 2004-2005 school year (Source: Educational Research Service) plus 25% estimated fringe benefits, 
unless otherwise noted. 

5 We assumed an original class size of 30 at all sites, although many districts reported higher class sizes in non Read 180 middle and high school language arts classes. 

6 No rec: no specific recommendation; depends on size of implementation and district resources. 

7 n.r.: not reported. 

8 Number of additional hours divided by 1,440 (work hours per year). This formula for personnel is used throughout the study. 

9 No national salary information was available. Teacher salary information was used as an estimate of in school technician costs. 

10 The cost of teacher time is calculated as part of teacher salaries. It also effects substitute time. 

11 Formula used to derive opportunity cost of training: Hours of training/ 1440 - % FTE. FTE* # trained - total training FTE. Total training FTE * annualized salary - 
opportunity cost of training. This calculation was used for all personnel opportunity costs. 

12 This district uses only “oversized" classrooms for Read 180. 

13 Annualized over three years based on reports from sites. 

14 Annualized over three years based on reports from sites. 
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TABLE n °- 2. Questioning the Author (QtA) Implementation Models 


INGREDIENTS LIST 

INGREDIENT 

UNIVERSITY OF PITTSBURGH 


Costs 

Annual 

Costs 

Suggested 

Inputs 17 

Total 

Costs 

Annual 

Costs 

PERSONNEL (FTE) 

Additional teachers required 18 

$57,355.0019 

$57,355.00 

0 

$0.00 

$0.00 

QtA teacher: student ratio 



1:20 20 



District teacher :student ratio 



1:20 



Students per teacher trained 



120 22 



School level Administration 

$105,282.50 

$105,282.50 

0 . 01 24 

$1,052.83 

$1,052.83 

District-level Administration 

$85,892.50 

$85,892.50 

0 

$0.00 

$0.00 

PROFFESIONAL DEVELOPMENT 

Teacher training 

$55.0025 

$12.71 

2 

$110.00 

$25.41 

Day 1 workshop 

$2,500.0026 

$577.50 

1 

$2,500.00 

$577.50 

Day 2 workshop 



0 



45 minute demonstration lesson 

included 


yes 



Observations - 8 per teacher 

$187.50 

$43.31 

16 

$3,000.00 

$693.00 

OR District Facilitator 

$85,892.50 

$19,841.17 

0.04 27 



Monthly meetings, annually 

included 


18 hours 28 



Total training days per teacher 



4.25 



Substitute days first year 30 

$120. 00 31 

$27.72 

0 



Planning time first year 



1.5 hrs per lesson 



Planning time after first year 



45 min per lesson 



School administrator training time 

$105,282.50 

$24,320.26 

0.005 

$526.41 

$121.60 

District administrators training time 

$85,892.50 

$19,841.17 

0.005 

$429.46 

$99.21 

FACILITIES 

Classrooms 



2 



EQUIPMENT/MATERIALS 



standard 



Total Cost 




$7,618.70 

$2,569.54 

Students served 34 




240 

240 

Cost Per Student 




$31.74 

$10.71 
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TABLE n °- 2. Questioning the Author (QtA) Implementation Models (continued) 


INGREDIENTS LIST 

QtA SITE ONE 15 

15 School district with 3,200 total enrollment. 

16 Provided by the Director of Curriculum and 







Total 

Annual 

Instruction. 


Inputs 16 

Costs 

Costs 

17 Provided by Dr. Margaret Mckeown, University of 




Pittsburgh. 

PERSONNEL (FTE) 

18 FTE: Full time employees. Costs for teacher time 
were only calculated above and beyond inputs 

Additional teachers required 18 


$0.00 

$0.00 

before the implementation of QtA. 

0 

19 All personnel costs are calculated using national 





averages for the 2004-2005 school year plus 25% 

QtA teacher: student ratio 

01:22. 5 21 



for fringe benefits unless otherwise noted. 

20 Average of the suggested class size of 12 to 28. 

District teacher :student ratio 

01:22.5 



21 Average of reported class size of 20 to 25. 

22 20 students per period, six periods per day. 

Students per teacher trained 

22. 5 23 



23 These teachers were multi-subject classroom 





teachers who stayed with the same students all day. 

School level Administration 

0 

$0.00 

$0.00 

24 Calculated by dividing the number of hours spent 





on QtA by the estimated total number of hours 

District-level Administration 

0.03 

$2,576.78 

$2,576.78 

worked, 1440. Similar calculations are made using 
the same method throughout the table. 

PROFFESIONAL 




25 Cost of teacher materials for professional 




development provided by Dr. Margaret Mckeown, 

DEVELOPMENT 




University of Pittsburgh. 





26 Flat rate for training provided by Dr. Margaret 

Teacher training 

25 

$1,375.00 

$317.63 

Mckeown, University of Pittsburgh. 

27 About 3 hours per month per QtA teacher. 

Day 1 workshop 

1 

$2,500.00 

$577.50 

28 1 hour per teacher per month. 

29 1 full day (8 hours) per teacher per month. 

Day 2 workshop 

1 



30 The substitute days may differ from total training 





days because some training took place over the 

45 minute demonstration lesson 

no 



summer. 





31 This is an estimate that we used throughout the 

Observations - 8 per teacher 

200 

$37,500.00 

$8,662.50 

study. Substitute costs vary substantially by region. 
32 Standard for the district. 

OR District Facilitator 

not used 



33 Standard for the district. 

34 Students per class* class periods per day - 

Monthly meetings, annually 

1800 hours 29 



students served per year * 5 - students served over 
the five year implementation period. 

Total training days per teacher 

12 




Substitute days first year 30 

225 

$27,000.00 

$6,237.00 


Planning time first year 

45 min 32 




Planning time after first year 

45 min 33 




School administrator training time 

0 

$0.00 

$0.00 


District administrators training time 

0.183 

$15,718.33 

$3,630.93 


FACILITIES 


Classrooms 

25 




EQUIPMENT/MATERIALS 

standard 




Total Cost 


$86,670.10 

$22,002.33 


Students served 34 


625 

625 


Cost Per Student 


$138.67 

$35.20 
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TABLE Vo. 3 . Reading Apprenticeship (RA) Implementation Models 

INGREDIENTS LIST 

INGREDIENT 

RA SITE ONE 


Cost 35 

Annualized 36 

Inputs 37 

Cost 

Annualized 

PERSONNEL 

Additional Teachers for RA 

$57,355.00 

$57,355.00 

0 

$0 

$0 

RA teacheristudent ratio 



1:22 



District teacher:student ratio 



1:22 



School level Administration 

$105,282.50 

$105,282.50 

0.01 39 

$1,052.83 

$1,052.83 

District-level Administration 

$85,892.50 

$85,892.50 

0.05 40 

$4,294.63 

$4,294.63 

PROFFESIONAL DEVELOPMENT 

1st Yr Teachers Trained 






NIRA 42 

$4,000.00 

$924.00 

8 

$33,000 43 

$7,623.00 

West Ed Site-based training 44 from: 






One Day 

$7,500.00 

$1,732.50 




Seven Days 

$50,000.00 

$11,550.00 




District Customized 



42 

$60,000 

$13,860.00 

Bay Area Network Series 

$ 1 , 000.00 

$231.00 

no 



Paid Collaboration Time 46 



yes 

$ 6 , 000.00 

$ 6 , 000.00 

Training after 1st Year 






Site-based training 

included 


yes 

included 


Continuing Network Series 

included 


no 



Substitute days 

$ 120.00 

$27.72 

2344 s 

$28,080.00 

$5,897.00 

School administrator training 

$105,282.50 

$24,320.26 

0.004 50 

$421.13 

$97.28 

District administrator training 

$85,892.50 

$19,841.17 

0.009 52 

$773.03 

$178.56 

Total Cost 




$133,621.62 

$39,003.30 

Students Served in ‘04-’05 




1,271 

1,271 

Cost per student 




$105.13 

$30.69 
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TABLE n». 3. Reading Apprenticeship (RA) Implementation Models (continued) 


INGREDIENTS LIST 

RA SITE TWO 

35 


Inputs 38 

Cost 

Annualized 


PERSONNEL 

36 

Additional Teachers for RA 

0 

$0 

$0 


RA teacherstudent ratio 

1:26 



37 

38 

District teacher:student ratio 

1:26 



39 

School level Administration 

0.01 

$1,052.83 

$1,052.83 

41 

District-level Administration 

0.07 41 

$5,964.76 

$5,964.76 


PROFFESIONAL DEVELOPMENT 

42 

1st Yr Teachers Trained 




43 

NIRA 42 

no 



44 

West Ed Site-based training 44 from: 

no 




One Day 




45 

Seven Days 




46 

District Customized 




48 

Bay Area Network Series 

445 

$4,000.00 

$924.00 


Paid Collaboration Time 46 

n.r 47 



50 

Training after 1st Year 




51 

Site-based training 

no 



52 

Continuing Network Series 

4 

included 


53 

Substitute days 

28 49 

$3,360.00 

$776.16 


School administrator training 

0.04 51 

$4,211.30 

$972.81 


District administrator training 

0.04 53 

$3,340.26 

$771.60 


Total Cost 


$21,929.15 

$10,462.16 


Students Served in ‘04-’05 


1,150 

1,150 


Cost per student 


$19.07 

$9.10 



All personnel costs were calculated using 
national averages for the 2004/5 school year 
(Source: Educational Research Service) plus 
25 %estimated fringe benefits. 0AII program 
costs are provided by WestEd. 

To annualize costs, this table assumes a five 
year lifespan and a five percent discount rate 
unless otherwise noted. 

Provided by assistant district superintendent. 
Provided by assistant district superintendent 
and school principal. 

Twenty hours per year. 

Seventy hours per year. 

One hundred hours per year: includes six 
hours of staff training meetings per month and 
forty hours of marketing, securing resources, 
presenting, organizing, etc. 

National Institute in Reading Apprenticeship. 
See www.wested.org/cs/sli for more 
information. 

$4,125 per teacher. This number includes 
travel expenses. 

For a complete description of these options, 
see http://www.wested.org/cs/sli/print/docs/ 
sli/services.htm. 

Twenty teachers have been trained over five 
years at approximately four per year. 

District pays teacher leaders for collaborative 
monthly meetings. 

Not reported. 

Five days per teacher for 42 teachers for the 
site based training and 3 days per teacher for 
8 teachers for NIRA training. 

Seven days per teacher for 4 teachers. 

Six hours total: four hrs % day training and two 
hrs end of year meeting. 

56 hours per administrator. One administrator 
participates in the seven day training each 
year. 

Participation in school administrator half-day 
training. 

District administrator participates in half-day 
training. 
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Endnotes 


14 Both Site Five administrators emphasized that they went 
above and beyond the requirements for successful RA 
implementation. 

Change-Agent studies. 

2 For further information on how to annualize costs, see Levin 
& McEwan (2001, pp. 67-70). 

3 The annualized value of additional classrooms is not 
included here on the assumption that if only a small 
proportion of students were enrolled in Read 180, space 
might be found for 90 minutes a day. However, if larger 
numbers were enrolled and schools were fully utilized, our 
assumption will understate the costs. 

4 In both this case and that of site three we questioned the 
large class sizes reported to us. Respondents replied that 
in both cases the districts were growing so rapidly that 
school construction could not keep up with the expansion of 
enrollments, resulting in very large classes at certain grade 
levels. 

5 Since class size remained at 30 (double the Read 180 
recommendation), there was no additional need for teachers 
to reduce class size. 

6 Scholastic views these costs as discretionary on the part of 
the district. The new version of Read 1 80 will provide for a 
centralized data processing and analysis system. 

7 The cost estimates in Table 1 are on the conservative side 
because we did not include several areas of potential cost 
including the annualized costs of extra classrooms, where 
needed. 

8 We have used a 5 year expected lifespan consistently 
throughout this study 

9 Estimated assuming a substitute teacher costs $120 per day 

10 For full descriptions of RA professional development 
options, see http://www.wested.org/cs/sli/view/serv. 

1 1 Estimated assuming a substitute teacher cost of $120 per day. 

12 The teacher-leaders trained in 2004-200S joined other 
teacher-leaders trained by SLI in previous years 

13 Estimated assuming a substitute teacher cost of $120 per day. 


1 The literature on resistance to school change and on how 
schools shape reforms (rather than reforms shaping schools) 
is substantial. On the former, see Evans (1996). On the 
latter, see the summary in McLaughlin (1990) of the Rand 
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